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Introduction 


Optical  microscopy  is  one  of  the  most  widely  used  methods  in  a  diverse  range  of  advanced  research,  industrial,  and 
clinical  settings  including  microelectronics,  food  science,  oceanography,  archaeology,  environmental  science,  energy 
science,  microbiology,  mineralogy,  and  pathology.  High  information  content  spatial  metrics  provided  by  imaging  are 


used  to  characterize  microscopic  particles  such  as  emulsions, 
microorganisms,  and  cells  in  particle  synthesis,  ecosystem 
monitoring,  biofuel  formulation,  drug  discovery, 
histopathology,  and  cytology-based  diagnostics.  An 
impressive  arsenal  of  optical  nanoscopy  methods  enables 
imaging  of  cellular  structures  beyond  the  diffraction  limit  of 
light.  Unfortunately,  while  there  has  been  breathtaking 
progress  in  improving  spatial  resolution  in  the  last  two 
decades,  mostly  overlooked  is  the  temporal  resolution  of 
imaging  systems. 

While  useful  for  visual  inspection  of  many  individual 
microparticles  without  human  intervention,  conventional 
automated  microscopy  does  not  have  high  throughput  and  is 
hence  unable  to  evaluate,  analyze,  and  screen  large 
populations  with  high  statistical  accuracy.  High- throughput 
detection  is  essential  to  rapidly  assay  morphological  and 
biochemical  properties  in  a  reasonable  period  of  time. 
Unfortunately,  the  throughput  of  state-of-the-art  automated 
microscopes  is  only  up  to  -1000  particles/s,  limiting  the 
particle  count.  As  a  benchmark,  this  throughput  is 
significantly  lower  than  that  of  single-pixel  high-throughput 
flow  analyzers  such  as  electronic  particle  counters  and  flow 
cytometers  (-100,000  particles/s)  which  trade  increased 
throughput  for  a  lack  of  spatial  resolution.  Because  of  this 
trade-off,  such  systems  are  often  unable  to  resolve  single, 
multiple,  and  clustered  particles  or  unusually  shaped  particles 
as  well  as  to  distinguish  debris  and  nonspecific  labeling  and 
hence  produce  a  large  number  of  false  positive  events  or 
inaccurate  subpopulation  counts. 
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Fig.  1.  STEAM  flow  analyzer.  An  animated  video  that  shows  the 
integrated  functionality  of  the  system  is  available.  (A)  Schematic  of 
the  STEAM  flow  analyzer  that  highlights  the  optical  layout  of  the 
STEAM  camera  and  real-time  optoelectronic  time-stretch  image 
processor.  The  STEAM  camera  takes  blur-free  images  of  fast¬ 
flowing  particles  in  the  microfluidic  device.  The  acquired  images 
are  optoelectronically  processed  and  screened  in  the  real-time 
optoelectronic  time-stretch  image  processor.  (B)  Microfluidic 
device  in  which  particles  are  controlled  to  flow  at  a  uniform 
velocity  and  focused  and  ordered  by  inertial  lift  forces  in  the 
microfluidic  channel.  (C)  Field-programmable  digital  image 
processor  that  captures  particles  and  performs  fully  automated 
particle  classification  in  real  time.  It  consists  of  (i)  a  high-speed 
analog-to -digital  converter  (ADC),  (ii)  a  field-programmable  gate 
array  (FPGA)  for  particle  capture,  E-slide  generation,  and  coarse 
particle  classification,  (iii)  an  on-board  memory  circuit  for  storing 
selected  E-slides,  and  (iv)  a  central  processing  unit  (CPU)  for  fine 
oarticle  classification. 


While  high-end  digital  cameras  [i.e.,  charge-coupled  device  (CCD)  and  complementary  metal-oxide-semiconductor 
(CMOS)  cameras]  are  now  able  to  perform  imaging  at  a  speed  approaching  1  million  frames/s,  they  are  not  well  suited 
for  high-throughput  microscopy.  First,  a  critical  limitation  that  hampers  imaging  of  particles  flowing  at  high  speeds  is 
the  relatively  long  shutter  speed  or  exposure  time  (>1  microsecond)  that  results  in  motion  blur  and  loss  of  resolution 
during  the  high-speed  flow,  hence  degrading  specificity.  Another  limitation  is  the  time  needed  to  download  the  image 
from  the  array  of  thousands  of  pixels.  To  achieve  high  frame  rates,  the  number  of  pixels  that  are  employed  must  be 
reduced  in  a  process  known  as  partial  readout.  The  penalty  is  that  image  resolution  is  lost  at  high  frame  rates. 
Furthermore,  there  is  a  fundamental  trade-off  between  sensitivity  and  speed  -  at  high  frame  rates,  fewer  photons  are 
captured  during  each  frame,  leading  to  reduction  in  sensitivity.  Finally,  currently  digital  image  processing  cannot  be 
performed  in  real  time  due  to  the  storage  and  access  of  the  massive  amount  of  digital  data  that  is  produced  during  high¬ 
speed  imaging  and  hence  requires  many  days  of  off-line  digital  processing.  This  translates  into  a  limited  particle  count, 
thus  compromising  statistical  accuracy  in  high-throughput  screening.  These  limitations  prevent  automated  microscopy 
from  analyzing  a  large  population  of  particles  with  high  statistical  accuracy  in  a  practical  period  of  time. 


To  address  the  needs  and  fill  in  the  technological  gap 
between  the  automated  microscope  and  high-throughput 
flow  analyzer,  we  propose  and  demonstrate  an  automated 
flow-through  single-particle  optical  microscopy  system  that 
can  evaluate,  analyze,  and  screen  a  large  population  of 
particles  with  high  specificity,  high  sensitivity,  and  high 
statistical  accuracy  in  a  short  time.  This  method  builds  on 
an  unique  integration  of  (i)  an  ultrafast  optical  imaging 
modality  known  as  serial  time-encoded  amplified 
microscopy  (STEAM)  for  blur-free  imaging  of  particles  in 
high-speed  flow,  (ii)  inertial  microfluidic  technology  for 
sheath-free  focusing  and  ordering  of  particles  with  inertial 
forces,  and  (iii)  hybrid  optoelectronic  image  processing 
circuitry  for  real-time  image  processing.  The  integrated 
system  transforms  particles  in  well-controlled  microfluidic 
flow  into  a  series  of  E-slides  -  an  electronic  version  of  glass 
slides  -  on  which  particles  of  interest  are  digitally  analyzed. 
With  the  power  of  optoelectronic  communication  and 
information  technologies,  this  property  enables  fully 
automated  real-time  image-recording  and  classification  of  a 
large  number  of  particles  through  their  morphological  and 
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Fig.  2.  Performance  of  the  STEAM  camera  and  comparison  with  a 
conventional  CCD  camera  and  a  state-of-the-art  CMOS  camera.  E- 

slides  of  flowing  particles  of  various  species  in  the  microfluidic  device 
were  generated  by  the  STEAM  flow  analyzer  with  the  built-in  STEAM 
camera  (27  ps  shutter  speed,  128  x  512  pixels,  25  dB  optical  image 
gain).  The  E-slides  are  compared  with  images  of  the  same  particles 
captured  by  a  state-of-the-art  CMOS  camera  (1  microsecond  shutter 
speed,  32  x  32  pixels,  no  optical  image  gain)  under  the  same  flow.  To 
operate  at  these  high  speeds,  the  CMOS  camera  used  partial  readout, 
limiting  the  number  of  pixels  to  32  x  32.  Here  the  particles  were 
controlled  to  flow  at  a  uniform  speed  of  4  m/s,  which  corresponds  to  a 
throughput  of  100,000  particles/s  based  on  the  volumetric  flow  rate.  The 
high-speed  motion  of  the  particles  was  frozen  by  the  ultrafast  shutter 
speed  (ultrashort  exposure  time)  of  the  STEAM  camera  (27  ps)  yet 
without  sacrificing  sensitivity  due  to  the  optical  image  amplification 
whereas  the  reduced  sensitivity  and  motion  blurs  caused  by  the  CMOS 
camera’s  much  lower  shutter  speed  and  lack  of  optical  image 
amplification  are  evident.  For  further  comparison,  stationary  particles  of 
the  same  types  on  a  glass  slide  were  obtained  under  a  conventional 
microscope  with  a  CCD  camera  with  a  much  longer  exposure  time  (17 
ms  shutter  speed,  1280  x  1024  pixels,  no  optical  image  gain).  Despite 
the  fact  that  the  STEAM  camera  is  many  orders  of  magnitude  faster 
than  the  CCD  camera,  the  two  cameras  share  similar  image  quality  (i.e., 
sensitivity  and  resolution). 


biochemical  features.  To  demonstrate  our  technology’s  utility,  we  show  real-time  non-stop  image-based  identification 
and  screening  of  budding  yeast  cells  at  different  budding  stages  and  rare  breast  cancer  cells  in  blood  with  an 
unprecedented  throughput  of  100,000  particles/s  and  a  record  false  positive  rate  of  one  in  a  million. 

Results 

Principle  of  the  high-throughput  imaging  flow  analyzer.  Our  system,  which  we  refer  to  as  the  STEAM  flow  analyzer 
(Fig.  1A),  consists  of  three  subsystems:  (i)  the  microfluidic  device  (Fig.  IB),  (ii)  the  STEAM  camera  (Fig.  1A),  and 
(iii)  the  real-time  optoelectronic  time-stretch  image  processor  (Fig.  1C).  An  animated  video  shows  the  integrated 
functionality  of  the  system.  The  STEAM  flow  analyzer  operates  in  three  steps.  First,  particles  are  controlled  to  flow  at  a 
uniform  velocity  and  focused  and  ordered  by  inertial  lift  forces  in  the  microfluidic  channel.  Second,  the  STEAM 
camera  takes  images  of  the  fast-flowing  particles.  Finally,  the  real-time  optoelectronic  time-stretch  image  processor 
processes  the  images  optically  and  electronically  and  then  performs  automated  particle  screening  in  real  time. 

The  STEAM  flow  analyzer  employs  inertial  microfluidic  technology  (Fig.  IB).  This  method  provides  uniform  positions 
and  velocities  for  particles  through  intrinsic  inertial  lift  forces  (important  for  generation  of  clear  E-slides)  while 
eliminating  the  need  for  sheath  fluid  which  is  traditionally  required  for  hydrodynamic  focusing  in  conventional  flow 
cytometers.  While  inertial  forces  are  typically  assumed  to  be  negligible  in  microfluidic  systems,  they  can,  in  fact,  be 
important  when  the  flow  rate  in  microchannels  is  high.  Particles  in  finite-inertia  confined  channel  flows  focus  to  cross- 
sectional  dynamic  equilibrium  positions  that  correspond  with  the  fold  of  symmetry  of  the  channel  cross  section. 
Furthermore,  longitudinal  ordering  (i.e.,  stabilized  interparticle  or  intercell  spacing  along  the  length  of  the  channel) 
results  from  inertial  lift  forces  and  the  flow  field  acting  together.  In  some  of  our  previous  work,  we  determined  an 
optimal  balance  between  inertial  lift  forces  and  Dean  drag  forces,  counter-rotating  vortices  perpendicular  to  primary 
channel  flow,  to  accurately  position  a  range  of  particle  sizes  to  a  single  streak  with  two  heights  within  the  channel  at  a 
high  flow  rate.  Here  we  reduced  the  length  of  one  of  these  optimized  channels  to  decrease  its  pressure  and  lowered  the 
height  of  the  channel  to  bring  the  distinct  equilibrium  positions  in  the  vertical  plane  into  closer  proximity.  To  ensure 
stability  in  real-time  image  processing,  the  microfluidic  device  was  fabricated  by  replica  molding  in  thermoset 
polyester  due  to  its  high  stiffness  and  ability  to  sustain  high  pressures. 
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The  STEAM  camera  first  captures  fast  sequential 
images  with  laser  pulses  and  then  stretches  image- 
encoded  pulses  in  time  so  that  they  can  be  digitized  and 
processed  in  real  time  (See  Materials  and  Methods). 
During  the  time-stretch  process,  images  are  also 
optically  amplified  to  overcome  the  thermal  noise 
inherent  in  photon-to-electron  conversion.  In  the  first 
step,  a  pair  of  diffraction  gratings  spatially  disperses  the 
broadband  optical  pulse  into  a  rainbow  designed  to 
capture  ID  line  scans  of  particles  flowing  through  the 
channel.  The  temporal  duration  of  these  pulses  is  27  ps 
(shutter  speed)  and  they  occur  at  a  repetition  rate  of 
36.7  MHz  (line  scan  rate).  The  back-reflected  pulses 
from  the  microfluidic  device  are  directed  via  an  optical 
circulator  toward  the  real-time  optoelectronic  time- 
stretch  image  processor  followed  by  the  high-speed 
photodetector  and  the  field- programmable  digital  image 
processor.  2D  E-slides  are  then  constructed  from  the 
digitized  ID  frames  and  made  available  for  screening. 

As  shown  in  Fig.  1A,  the  real-time  optoelectronic  time- 
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Fig.  3.  High-throughput  screening  of  budding  yeast  with  the  STEAM 
flow  analyzer.  (A)  Screening  process  of  the  field-programmable  digital 
image  processor.  The  FPGA  performs  cell  capture  while  ignoring  the  blank 
images  between  cells  and  stores  the  corresponding  E-slides  into  the  on-board 
memory.  The  CPU  then  distinguishes  between  budding  and  unbudded  cells, 
classifies  the  budding  cells  by  daughter-to-mother  ratio  in  size,  and  finally 
generates  a  histogram  for  the  subpopulations.  (B)  Subpopulation  analysis  of 
yeast  at  different  stages  of  budding.  The  total  number  of  captured  yeast  cells 
is  75509,  about  34%  of  which  constitutes  budding  cells.  Here  the  entire 
procedure  that  consists  of  the  measurement,  image  analysis,  and  histogram 
generation  takes  less  than  a  few  minutes.  Since  the  STEAM  flow  analyzer 
operates  in  real  time,  time-resolved  statistical  analysis  of  the  subpopulations 
can  also  be  performed  to  control  and  optimize  the  budding  process. 
Furthermore,  similar  capture  and  subpopulation  analysis  can  also  be  applied 
to  emulsions  for  applications  in  cosmetics  and  pharmaceutics. 


stretch  image  processor  consists  of  an  amplified  dispersive  Fourier  transformer,  a  high-speed  photodetector,  and  a 


field-programmable  digital  image  processor.  First,  the  amplified  dispersive  Fourier  transformer  serializes  the  image- 
encoded  pulse  (i.e.,  the  ID  image  frame)  into  a  pixel  stream  that  is  slowed  down  in  time  so  that  it  can  be  digitized  and 


processed  in  real  time.  While  it  is  being  stretched  in  time,  the  image-carrying  pulse  is  amplified  in  the  optical  domain 
through  distributed  Raman  amplification  in  the  time-stretch  element.  Raman  amplification  is  chosen  as  it  has  a  broad 
and  flexible  optical  spectrum  and  also  leads  to  low  noise  figure  in  the  frequency-to-time  mapping  process.  The  optical 


image  amplification  (~1000x)  before  photon-to-electron  conversion  is  critical  as  it  enables  high-throughput  microscopy 
in  the  ultrashort  exposure  time  (27  ps)  without  the  need  for  a  high- intensity  illuminator.  In  addition,  the  ultrafast  shutter 


speed  freezes  any  motion  of  particles  in  high-speed  flow,  allowing  for  acquisition  of  blur-free  images.  The  amplified 
ID  data  stream  is  detected  by  the  high-speed  single-pixel  photodetector,  eliminating  the  need  for  a  detector  array  and 


hence  readout  time  limitations.  After  analog-to-digital  conversion  by  a  real-time  digitizer,  the  data  is  processed  in  real 
time  in  the  field-programmable  digital  image  processor  with  a  programmable  multi-stage  decision-making  architecture 


(Fig.  1C)  that  employs  a  field-programmable  gate  array  (FPGA),  an  on-board  memory  circuit,  and  a  central  processing 
unit  (CPU).  Specifically,  the  function  of  the  FPGA  is  to  identify  the  presence  of  particles  (ignoring  the  unimportant 
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space  between  the  particles),  generate  E- slides  for  the  particles,  coarsely  down-select  the  slides  (i.e.,  the  particles) 
through  their  morphological  and  biochemical  markers,  and  store  those  selected  in  the  memory.  As  a  second-step 
screening  process,  the  CPU  performs  fine  particle  classification  on  the  stored  slides  with  more  stringent  algorithms 
such  as  decision-tree  classification  and  supervised  learning  methods. 

The  STEAM  flow  analyzer’s  blur-free  image  acquisition  enables  differentiation  of  particles  from  a  heterogeneous 
population.  Fig.  2  shows  E-slides  of  fast-flowing  particles  of  various  species  in  the  microfluidic  channel  captured  by 
the  STEAM  flow  analyzer.  Here  the  particles  were  controlled  to  flow  at  a  uniform  speed  of  4  m/s,  which  corresponds  to 
a  throughput  of  100,000  particles/s  -  a  very  fast  flow,  but  motion  blur  is  absent  in  the  images  due  to  the  ultrafast  shutter 
speed  of  the  STEAM  camera  (27  ps).  For  comparison,  Fig.  2  also  shows  images  of  the  same  types  of  particles  under  the 
same  flow  conditions  captured  by  a  state-of-the-art  CMOS  camera.  These  images  show  that  the  CMOS  camera’s  lower 
shutter  speed  ( 1  microsecond)  and  lack  of  optical  image  amplification  significantly  reduced  the  sensitivity  and  caused 
motion  blur  in  the  images,  rendering  the  camera  unable  to  classify  particles  reliably.  For  further  comparison,  stationary 
particles  of  the  same  types  on  a  glass  slide  were  obtained  under  a  conventional  microscope  with  a  CCD  camera  with  a 
much  longer  exposure  time  (shutter  speed)  of  17  ms  (Fig.  2).  Despite  the  fact  that  the  STEAM  camera  is  many  orders 
of  magnitude  faster  than  the  CCD  camera,  the  two  cameras  share  similar  image  quality  (i.e.,  sensitivity  and  resolution). 

Screening  of  budding  yeast  with  the  high-throughput  imaging  flow  analyzer.  To  show  the  utility  of  the  STEAM  flow 
analyzer,  we  used  it  to  demonstrate  high-throughput  screening  of  Saccharomyces  cerevisiae,  commonly  known  as 
budding  yeast.  Budding  yeast  are  important  in  food  science,  are  used  as  a  model  for  studying  eukaryotic  cell  biology, 
and  are  engineered  to  produce  protein  therapeutics.  Growth  in  yeast  can  be  studied  and  optimized  by  flow  cytometry  or 
microscopy  -  both  of  which  possess  specific  limitations.  Biochemical  analysis  alone  via  flow  cytometry  is  insufficient 
for  analysis  of  complex  checkpoints  and  detection  of  minor  perturbations  in  the  cell  cycle  and  provides  poor 
characterization  of  the  asymmetric  growth  of  yeast  in  comparison  with  imaging.  A  high-throughput  microscopy 
technique  such  as  our  STEAM  flow  analyzer  could  provide  a  powerful  screening  tool  to  assay  morphological  changes 
potentially  accompanying  systematic  gene  knockouts  or  other  molecular  perturbations. 
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The  STEAM  flow  analyzer  can  efficiently  identify  and  screen  budding  yeast  cells  and  perform  subpopulation  analysis 
in  real  time.  Here  we  programmed  the  FPGA  and  CPU  to  capture  incoming  cells,  distinguish  between  budding  and 
unbudded  cells,  and  classify  budding  cells  based  on  different  budding  stage  (Fig.  3A).  Specifically,  the  FPGA  performs 
cell  capture  while  ignoring  the  blank  images  and  stores  the  corresponding  E-slides  in  the  on-board  memory  (1  GB) 
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Fig.  4.  Rare  cell  detection  with  the  STEAM  flow  analyzer.  (A)  Scatter  plots  of  white  blood  cells  and  MCF7  cells  based  on  the  cell  size  (i.e.,  area)  and 
the  presence  of  the  surface  antigens  (i.e.,  metal  beads).  The  total  number  of  events  is  -10,000  for  both  cell  types.  This  statistical  analysis  is  used  to  build  a 
model  and  train  the  supervised  learning  method  and  hence  the  algorithms  to  run  the  field-programmable  digital  image  processor.  The  threshold  for  the 
size-based  selection  performed  on  the  FPGA  is  set  such  that  smaller  MCF7  cells  are  also  selected  at  the  expense  of  detecting  larger  white  blood  cells.  The 
rare  white  blood  cell  events  that  overlap  with  the  distribution  of  MCF7  cells  are  doublets  or  clusters  and  can  hence  be  rejected  by  imaging  (which  is  not 
possible  with  single-point  detection  methods).  (B)  Selection  process  of  the  field-programmable  digital  image  processor.  The  FPGA  performs  cell  capture, 
coarse  size-based  classification,  and  filters  out  more  than  99.9995%  of  white  blood  cells  while  leaving  only  -100  false  positive  events  per  mL  of  lysed 
blood  along  with  true  positive  events.  The  CPU  then  performs  fine  classification  by  circularity  and  presence  of  metal  beads  and  further  down-selects  cells 
by  an  order  of  magnitude,  leaving  true  positive  events  and  -10  false  positive  events  per  mL  of  lysed  blood  which  arise  due  to  image  processing  artifacts 
which  can  further  be  rejected  by  human  visual  inspection.  (C)  Statistical  analysis  of  the  system’s  capture  efficiency  for  various  concentrations.  The  results 
indicate  that  the  field-programmable  digital  image  processor  can  identify  extremely  rare  cells  with  a  high  efficiency  of  75%  (limited  by  the  imperfect 
coating  efficiency  and  missing  smaller  MCF7  cells  in  the  FPGA  selection  process).  Here  all  the  measurements  were  performed  with  bead-coated  MCF7 
cells  spiked  in  buffer  containing  white  blood  cells  from  3  mL  of  lysed  blood  (-80  million  white  blood  cells)  at  a  throughput  of  100,000  cells/s.  Individual 
samples  were  measured  four  times,  establishing  that  the  classification  is  highly  repeatable  (indicated  by  the  vertical  error  bars).  Moreover,  the  correlation 
of  detected  MCF7  cells  with  spiked  MCF7  cells  is  good  (r2  =  0.94).  The  horizontal  deviations  can  be  attributed  to  several  known  sources  of  error  in  the 
spiking  method  including  the  initial  hemacytometer  count  .  (D)  ROC  curve  analysis  of  the  STEAM  flow  analyzer  in  comparison  with  the  conventional 
flow  cytometer.  Our  method  is  sufficiently  sensitive  for  detection  of  -1  MCF7  cell  in  a  million  white  blood  cells  (i.e.,  a  false  positive  rate  of  -10’6)  and  is 
two  orders  of  magnitude  better  in  terms  of  false  positive  rate  than  the  conventional  flow  cytometer  yet  without  sacrificing  throughput. 


which  can  save  up  to  ~16,000v  slides  (each  of  which  has  64/v  KB),  where  v  is  the  flow  speed  in  units  of  m/s.  Here 
since  the  2D  E-slides  are  constructed  from  a  series  of  ID  frames  with  the  second  dimension  provided  by  the  flow,  the 
size  of  each  slide  decreases  and  hence  the  total  number  of  slides  that  can  be  saved  in  the  memory  increases  as  the  flow 
speed  increases.  We  estimate  that  the  flow  speed  can  be  set  up  to  8  m/s  (corresponding  to  200,000  cells/s)  while 
retaining  reasonable  image  quality  and  classification  accuracy.  The  saved  E- slides  are  then  subject  to  the  CPU’s 
classification  process  in  which  the  budding  yeast  cells  are  isolated  and  classified  by  daughter-to-mother  ratio  in  size  to 
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generate  a  histogram  for  the  subpopulations  using  a  decision- tree  classification  method  (Fig.  3B).  For  comparison,  we 
also  performed  normal  microscopic  observation  of  yeast  cells  with  a  CCD  camera  and  performed  digital  image 
processing  similar  to  the  automated  classification  on  the  CPU  in  the  STEAM  flow  analyzer  and  found  good  agreement 
between  our  high-throughput  method  and  the  conventional  method,  suggesting  that  the  STEAM  flow  analyzer  is  useful 
for  analysis  of  yeast  growth  behavior  with  high  statistical  accuracy.  Since  the  STEAM  flow  analyzer  operates  in  real 
time,  time-resolved  statistical  analysis  of  the  subpopulations  can  also  be  performed  to  control  and  optimize  the  budding 
process.  Furthermore,  similar  capture  and  subpopulation  analysis  with  real-time  feedback  is  expected  to  be  useful  for 
optimization  of  emulsions  for  applications  in  cosmetics  and  pharmaceutics. 

Rare  cell  detection  with  the  high-throughput  imaging  flow  analyzer.  To  further  show  the  utility  of  the  STEAM  flow 
analyzer,  we  used  it  to  demonstrate  rare  cell  detection.  While  statistically  insignificant  and  hence  often  ignored,  rare 
events  among  a  large  heterogeneous  population  of  cells  in  blood  such  as  hematopoietic  stem  cells,  antigen- specific  T 
cells,  and  circulating  tumor  cells  are  important  in  biomedical  research  as  well  as  medical  diagnostics  and  therapeutics. 
Such  rare  cells  can  be  identified  by  a  combination  of  morphological  (i.e.,  size,  circularity,  and  clustering)  and 
biochemical  (i.e.,  surface  antigens)  markers.  Flere  our  model  for  rare  cells  is  the  MCF7  cell  line  (breast  cancer)  spiked 
in  blood.  Red  blood  cells  are  lysed  with  a  hypotonic  lysing  agent  while  MCF7  cells  are  fixed  with  formaldehyde  and 
coated  with  metal  beads  with  a  diameter  of  1  micron  via  an  antibody  to  EpCAM  (a  cell  surface  molecule  that  exists  on 
the  surface  of  epithelial  cells,  but  not  on  the  surface  of  blood  cells).  Our  observation  under  a  conventional  microscope 
indicates  that  -80%  of  MCF7  cells  are  coated  with  5-20  metal  beads. 

To  demonstrate  how  surface  marker  detection  could  be  achieved  and  coupled  with  morphological  analysis  in  our 
system,  we  performed  off-line  statistical  analysis  of  white  blood  cells  and  coated  MCF7  cells.  This  validation  is 
important  as  it  is  required  to  build  a  fully  trained  model  for  our  supervised  learning  method  or  support  vector  machine 
(SVM)  to  be  implemented  on  the  FPGA  and  CPU.  The  SVM  model  predicts  whether  a  new  target  (i.e.,  cell)  falls  into 
any  of  the  pre-defined  cell  categories  and  is  then  used  to  identify  the  target  cells  in  real  time.  Fig.  4A  shows  scatter 
plots  of  white  blood  cells  and  MCF7  cells  based  on  the  cell  size  (i.e.,  diameter)  and  presence  of  surface  antigens  (i.e., 
metal  beads)  produced  by  our  system.  The  plots  indicate  that  the  system  is  capable  of  differentiating  most  white  blood 
cells  and  MCF7  cells  with  high  specificity  using  a  single  molecular  marker. 

With  the  fully  trained  cell  classification  model,  the  STEAM  flow  analyzer  can  detect  MCF7  cells  as  rare  as  -1  in  a 
million  in  a  short  period  of  time.  Here  we  programmed  the  FPGA  and  CPU  such  that  the  FPGA  captures  incoming  cells 
and  performs  size-based  cell  classification  while  the  CPU  performs  classification  by  circularity  and  presence  of  metal 
beads.  The  threshold  for  the  size-based  selection  is  set  such  that  smaller  MCF7  cells  are  also  selected  at  the  expense  of 
detecting  larger  white  blood  cells  (Fig.  4A).  Yet,  this  process  efficiently  rejects  more  than  99.9995%  of  white  blood 
cells,  all  residual  red  blood  cells,  and  all  free-floating  metal  beads,  leaving  only  a  small  number  of  false  positive  events 
(-100  per  mL  of  lysed  blood)  along  with  true  positive  events  (Fig.  4B).  The  FPGA’s  ability  to  filter  out  the  large 
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number  of  blood  cells  and  avoid  storing  the  massive  amount  of  digital  data  enables  non-stop  real-time  operation.  The 
down-selected  E-slides  (i.e.,  cell  images)  are  stored  in  the  memory  (1GB)  which  can  save  up  to  ~4000v  slides  (each  of 
which  has  256 /v  KB),  where  v  is  the  flow  speed  in  units  of  m/s  as  described  above.  The  stored  slides  are  then  subject  to 
the  CPU’s  selection  process  in  which  the  cells  are  further  classified  by  circularity  and  presence  of  metal  beads.  This 
process  rejects  almost  all  the  false  positive  events  (i.e.,  multiple,  unfocused,  or  unordered  white  blood  cells)  that  have 
survived  from  the  initial  selection  process  on  the  FPGA  while  leaving  true  positive  events  and  a  very  small  number  of 
false  positive  events  (~10  per  mL  of  lysed  blood)  which  arise  due  to  image  processing  artifacts  and  can  further  be 
rejected  by  human  visual  inspection. 

Our  statistical  analysis  of  the  capture  efficiency  indicates  that  the  field-programmable  digital  image  processor  can 
identify  extremely  rare  cells  with  a  high  efficiency  of  75%  (limited  by  the  imperfect  coating  efficiency  and  missing 
smaller  MCF7  cells  in  the  FPGA  selection  process)  (Fig.  4C).  Furthermore,  our  receiver  operating  characteristic  (ROC) 
curve  analysis  of  the  results  indicates  that  our  method  is  sufficiently  sensitive  for  detection  of  ~1  MCF7  cell  in  a 
million  white  blood  cells  and  is  100  times  better  in  terms  of  false  positive  rate  than  the  conventional  flow  cytometer 
(Fig.  4D),  yet  without  sacrificing  throughput.  Here  all  the  measurements  were  performed  at  a  throughput  of  100,000 
cells/s,  corresponding  to  screening  of  10  mF  of  lysed  blood  in  less  than  15  minutes. 

Discussion 

We  have  developed  a  high-throughput  single-microparticle  flow-through  image  analyzer  for  real-time  image 
acquisition  and  screening  of  a  large  heterogeneous  population  of  particles.  By  overcoming  the  technological  limitations 
that  exist  in  conventional  automated  microscopes  (with  many  pixels  and  -1,000  particles/s  throughput)  and  high- 
throughput  flow  analyzers  (with  a  single  pixel  and  -100,000  particles/s  throughput),  our  system  has  achieved  real-time 
image-based  screening  with  high  sensitivity  (-75%),  high  specificity  (one  in  a  million),  and  high  statistical  accuracy 
(-100,000  particles/s)  simultaneously.  Here  the  throughput  is  only  limited  by  the  microfluidic  device’s  tolerance  to 
high  pressures  caused  by  high  flow  rates,  not  the  STEAM  camera’s  image  acquisition  speed  or  the  field- programmable 
digital  image  processor’s  processing  speed.  We  estimate  that  the  throughput  can  be  increased  to  200,000  particles/s 
while  retaining  reasonable  image  quality  and  classification  accuracy.  Our  method  can  also  be  combined  with  a 
conventional  cell  sorter  to  sort  out  rare  target  cells  for  further  genetic  analysis  of  the  cells.  Furthermore,  our  method 
can,  in  principle,  work  at  shorter  wavelengths  for  higher  spatial  resolution,  provided  that  proper  optical  components  are 
available. 

While  in  our  proof-of-concept  demonstration,  we  showed  high-throughput  screening  of  budding  yeast  and  detection  of 
rare  breast  cancer  cells  spiked  in  blood,  our  method  should  also  be  amenable  to  other  applications  in  which  high- 
throughput  microscopy  is  required.  Such  applications  include  imaging  and  detection  of  bioparticles  of  interest  in 
oceanography  (e.g.,  phytoplanktons),  energy  science  (e.g.,  oil  emulsions  and  engineered  microbes),  environmental 
science,  food  science,  cosmetics,  pharmaceutics,  and  medicine. 
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Key  Research  Accomplishments 

1.  Integration  of  optics,  microfluidics,  digital  and  analog  electronics,  and  algorithms  to  realize  a  ultrahigh  throughput 
automated  flow-through  microscope 

2.  Detection  of  rare  MCF7  breast  cancer  cells  in  lysed  blood  with  a  sensitivity  of  one  in  a  million  -  a  record  value  and 
one  that  represents  lOOx  improvement  over  the  current  state-of-the-art  Fluorescence  Activated  Cell  Sorter  (FACS). 

Reportable  Outcomes 

Realization  of  an  automated  high-throughput  flow-through  microscope  with  the  ability  to  detect  one  MCF7  breast 
cancer  in  a  million  white  blood  cells 

1.  B.  Mali,  K.  Goda,  and  K.  K.  Tsia,  “Apparatus  and  method  for  optically  amplified  imaging,”  US  2010/0141829  A1 

(2010) 

2.  B.  Mali,  K.  Goda,  and  K.  K.  Tsia,  “Apparatus  and  method  for  dispersive  Fourier-transform  imaging,”  US 
12/985539 (2010) 

3.  K.  Goda  et  al,  “High-throughput  single-microparticle  imaging  flow  analyzer,”  PNAS 
doi :  10. 1 073/pnas.  1 2047 1 8 1 09  (20 1 2) 

Conclusion 

In  summary,  we  have  developed  a  high-throughput  single-cell  flow-through  image  analyzer  for  real-time  image 
acquisition  and  screening  of  a  large  heterogeneous  population  of  cells  with  high  sensitivity  and  high  statistical 
accuracy.  The  throughput  is  limited  by  the  microfluidic  device’s  tolerance  to  high  pressures  caused  by  high  flow  rates, 
not  the  STEAM  camera’s  image  acquisition  speed  as  well  as  the  field-programmable  digital  image  processor’s 
processing  speed.  We  estimate  that  the  throughput  can  be  increased  to  200,000  cells/s  while  classification  accuracy. 
Our  method  can  also  be  combined  with  a  conventional  cell  sorter  to  sort  out  rare  cells  for  further  chemical  analysis  of 
the  cells.  While  in  our  proof-of-concept  demonstration,  we  showed  high-throughput  screening  of  budding  yeast  and 
detection  of  rare  breast  cancer  cells  in  blood,  our  method  should  also  be  amenable  to  other  applications  in  which  high- 
throughput  microscopy  is  required,  such  as  those  found  in  oceanography,  energy  science,  and  medicine. 
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